{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "private_outputs": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "## About this demo\n",
        "This demo describes step-by-step how to use BMF to develop a transcoding program, including video transcoding, audio transcoding, and image transcoding. In it, you can familiarize yourself with how to use BMF and how to use FFmpeg-compatible options to achieve the capabilities you need."
      ],
      "metadata": {
        "id": "4-zV1WWh4KHR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 1. Environmental preparation"
      ],
      "metadata": {
        "id": "zfRMFe8T7m7V"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1.1 FFmpeg\n",
        "FFmpeg 4.x or 5.x is needed by BMF when transcoding, check versions via apt:"
      ],
      "metadata": {
        "id": "91HYL6LrOpOS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "! apt show ffmpeg | grep \"^Package:\\|^Version:\""
      ],
      "metadata": {
        "id": "JRc_qwCp7lKQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "If the version meets the requirements, install ffmpeg via apt:"
      ],
      "metadata": {
        "id": "AtQwyUAs8Sjt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "! apt install -y ffmpeg"
      ],
      "metadata": {
        "id": "P2HXgryJ8WM1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Otherwise, you need to compile ffmpeg from source, you can use the script we provided(only linux and macos now):"
      ],
      "metadata": {
        "id": "UdubJ4ld-s1E"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "! git clone https://github.com/BabitMF/bmf bmf\n",
        "! ./bmf/scripts/build_ffmpeg.sh nasm yasm x264 x265"
      ],
      "metadata": {
        "id": "Fd_ig_h-DRqo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1.2 BMF\n",
        "BMF can be installed in many ways, we use pip here:"
      ],
      "metadata": {
        "id": "ozFWzDUlEZ_2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "! pip3 install BabitMF"
      ],
      "metadata": {
        "id": "nifZsafGEtAU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1.3 wurlitzer(optional)\n",
        "This package is installed to show the BMF C++ logs in the colab console, otherwise only python logs are printed. This step is not necessary if you're not in a Colab or iPython notebook environment."
      ],
      "metadata": {
        "id": "DZIEEZrsorvS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install wurlitzer\n",
        "%load_ext wurlitzer"
      ],
      "metadata": {
        "id": "SGUtzAwyo0A3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 2. Transcode demo"
      ],
      "metadata": {
        "id": "K6d5Zx1gE40p"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Download the video file we will be using first:"
      ],
      "metadata": {
        "id": "cWV4P15GpnxI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!gdown --fuzzy https://drive.google.com/file/d/1l8bDSrWn6643aDhyaocVStXdoUbVC3o2/view?usp=sharing -O big_bunny_10s_30fps.mp4"
      ],
      "metadata": {
        "id": "nvxgOt8upwVO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! ffprobe big_bunny_10s_30fps.mp4"
      ],
      "metadata": {
        "id": "4nnPCG_DuYA4"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 2.1 video transcoding"
      ],
      "metadata": {
        "id": "XGT6gZcmFfbU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 2.1.1 remuxing demo\n",
        "This demo shows how to demux a video in mp4 format and remuxing it into hls slices without involving audio and video decoding and encoding:\n"
      ],
      "metadata": {
        "id": "rYxE9jciqWQ3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import bmf\n",
        "\n",
        "input_video_path = \"./big_bunny_10s_30fps.mp4\"\n",
        "output_path = \"./remux_output.m3u8\"\n",
        "\n",
        "# create graph\n",
        "graph = bmf.graph()\n",
        "\n",
        "# decode\n",
        "video = graph.decode({\n",
        "    \"input_path\": input_video_path,\n",
        "    \"video_codec\": \"copy\",\n",
        "    \"audio_codec\": \"copy\"\n",
        "})\n",
        "\n",
        "(\n",
        "    bmf.encode(\n",
        "        video['video'],\n",
        "        video['audio'],\n",
        "        {\n",
        "            \"output_path\": output_path,\n",
        "            \"format\": \"hls\",\n",
        "            \"mux_params\": {\n",
        "                \"hls_list_size\": \"0\",\n",
        "                \"hls_time\": \"2\",\n",
        "                \"hls_segment_filename\": \"./file%03d.ts\"\n",
        "            }\n",
        "        }\n",
        "    ).run()\n",
        ")"
      ],
      "metadata": {
        "id": "6aqHpwuyWHDW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! ffprobe remux_output.m3u8\n",
        "! rm -rf remux_output.m3u8 file*.ts"
      ],
      "metadata": {
        "id": "yCgv91CmzgJg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 2.1.2 decoding and encoding demo\n",
        "This demo shows how to decode a H.264 video, drop the audio, scale the video to 720x576p resolution, and encode it with x265 encoder. You can even do it in one line of code:"
      ],
      "metadata": {
        "id": "kR0-53q0vJpW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import bmf\n",
        "\n",
        "input_video_path = \"./big_bunny_10s_30fps.mp4\"\n",
        "output_path = \"./decode_scale_encode_output.mp4\"\n",
        "(\n",
        "    bmf.graph()\n",
        "        .decode({'input_path': input_video_path})['video']\n",
        "        .scale(720, 576)\n",
        "        .encode(None, {\n",
        "            \"output_path\": output_path,\n",
        "            \"video_params\": {\n",
        "                \"codec\": \"libx265\"\n",
        "            }\n",
        "        }).run()\n",
        ")"
      ],
      "metadata": {
        "id": "P94Vb09JvN-G"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! ffprobe decode_scale_encode_output.mp4\n",
        "! rm -rf decode_scale_encode_output.mp4"
      ],
      "metadata": {
        "id": "2Px4bZmK4lWm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 2.1.3 multi stream demo\n",
        "This demo shows how to decode the original video, use the ffmpeg filter capability, and encode multi videos with different options:"
      ],
      "metadata": {
        "id": "AdqICSEi5DL0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import bmf\n",
        "\n",
        "input_video_path = \"./big_bunny_10s_30fps.mp4\"\n",
        "output_path0 = \"./decode_encode_multi_output0.mp4\"\n",
        "output_path1 = \"./decode_encode_multi_output1.mp4\"\n",
        "\n",
        "streams = bmf.graph().decode({'input_path': input_video_path})\n",
        "split_streams = streams['video'].split()\n",
        "bmf.encode(split_streams[0], streams['audio'], {\n",
        "    \"output_path\": output_path0,\n",
        "    \"video_params\": {\n",
        "        \"codec\": \"libx264\",\n",
        "        \"x264-params\": \"ssim=1:psnr=1\"\n",
        "    }\n",
        "})\n",
        "(\n",
        "    bmf.encode(split_streams[1], streams['audio'], {\n",
        "            \"output_path\": output_path1,\n",
        "            \"video_params\": {\n",
        "                \"codec\": \"libx265\",\n",
        "                \"preset\": \"fast\",\n",
        "                \"crf\": \"23\"\n",
        "            }\n",
        "        }).run()\n",
        ")"
      ],
      "metadata": {
        "id": "EB62DCjXDFf_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! ffprobe decode_encode_multi_output0.mp4\n",
        "! echo \"----------------------------------------------------------------------\"\n",
        "! ffprobe decode_encode_multi_output1.mp4\n",
        "\n",
        "! rm -rf decode_encode_multi_output0.mp4 decode_encode_multi_output1.mp4"
      ],
      "metadata": {
        "id": "DpxOsI0g6qrZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 2.2 audio transcoding\n"
      ],
      "metadata": {
        "id": "C0LlJz_IFUGd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "This demo shows how to use BMF to decode the input video, extract the audio part, add a piece of null audio before and after, and then encode the output wav file:"
      ],
      "metadata": {
        "id": "hdx0Fz95PX3B"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import bmf\n",
        "\n",
        "input_video_path = \"./big_bunny_10s_30fps.mp4\"\n",
        "output_path = \"./with_null_audio.wav\"\n",
        "\n",
        "# create graph\n",
        "graph = bmf.graph()\n",
        "\n",
        "# decode\n",
        "streams = graph.decode({\n",
        "    \"input_path\": input_video_path\n",
        "})\n",
        "\n",
        "# create a null audio stream\n",
        "audio_stream1 = graph.anullsrc('r=48000', 'cl=2').atrim('start=0', 'end=6')\n",
        "audio_stream2 = graph.anullsrc('r=48000', 'cl=2').atrim('start=0', 'end=6')\n",
        "concat_audio = (\n",
        "    bmf.concat(audio_stream1, streams['audio'], audio_stream2, n=3, v=0, a=1)\n",
        ")\n",
        "\n",
        "(\n",
        "    bmf.encode(\n",
        "        None,\n",
        "        concat_audio,\n",
        "        {\n",
        "            \"output_path\": output_path,\n",
        "            \"audio_params\": {\n",
        "                \"codec\": \"aac\",\n",
        "                \"bit_rate\": 128000,\n",
        "                \"sample_rate\": 44100,\n",
        "                \"channels\": 2\n",
        "            }\n",
        "        }\n",
        "    )\n",
        "    .run()\n",
        ")"
      ],
      "metadata": {
        "id": "z9zqj2leWHx1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! ffprobe with_null_audio.wav\n",
        "! rm -rf with_null_audio.wav"
      ],
      "metadata": {
        "id": "3SfQ0jyVlFyR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 2.3 image transcoding\n",
        "This demo shows how to decode a video, detect and skip black frames, encode and output to the pipeline, and then write files at the application layer. You can also do other custom operations, such as uploading directly to the cloud to avoid disk IO:"
      ],
      "metadata": {
        "id": "ZE0JZrkNFWFY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import bmf\n",
        "\n",
        "input_video_path = \"./big_bunny_10s_30fps.mp4\"\n",
        "\n",
        "graph = bmf.graph()\n",
        "streams = graph.decode({\n",
        "    \"input_path\": input_video_path,\n",
        "})\n",
        "video = streams['video'].ff_filter(\"blackframe\", threshold=32).ff_filter(\"metadata\", \"select:key=lavfi.blackframe.pblack:value=100:function=less\")\n",
        "vframes_num = 2\n",
        "result = (\n",
        "    bmf.encode(\n",
        "        video,\n",
        "        None,\n",
        "        {\n",
        "            \"push_output\": 1,\n",
        "            \"vframes\": vframes_num,\n",
        "            \"format\": \"image2pipe\",\n",
        "            \"avio_buffer_size\": 65536, #16*4096\n",
        "            \"video_params\": {\n",
        "                \"codec\": \"jpg\",\n",
        "                \"width\": 640,\n",
        "                \"height\": 480\n",
        "            },\n",
        "        }\n",
        "    )\n",
        "    .start()\n",
        ")\n",
        "write_num = 0\n",
        "for i, packet in enumerate(result):\n",
        "    avpacket = packet.get(bmf.BMFAVPacket)\n",
        "    data = avpacket.data.numpy()\n",
        "    if write_num < vframes_num:\n",
        "        output_path = \"./simple_image\" + str(write_num)+ \".jpg\"\n",
        "        write_num = write_num + 1\n",
        "        with open(output_path, \"wb\") as f:\n",
        "            f.write(data)"
      ],
      "metadata": {
        "id": "AUcjjyR5WM25"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! ffprobe simple_image0.jpg\n",
        "! rm -rf simple_image*.jpg"
      ],
      "metadata": {
        "id": "xmW17RiI-Fid"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}